Skip to content

Merge upstream DeepGEMM through PR #432 - #15

Merged
murphymatt merged 4 commits into
mainfrom
sync-upstream-pr-432
Sep 12, 2026
Merged

murphymatt merged 4 commits into
mainfrom
sync-upstream-pr-432

Conversation

@murphymatt

@murphymatt murphymatt commented Sep 10, 2026 •

Copy link
Copy Markdown

Merge upstream DeepGEMM through 66081d4c9c7d7c44f13fea402e5b622aa0f409c2 (PR #432), including the July release and tensormap race fix. The merge retains all six fork commits and ports the fork's training paths onto upstream's DeepJIT runtime, buffer layout, and scheduler.

  • Preserve GeGLU, BF16/MXFP4 backward kernels, activation saves across ring wraps, route-weight/combine modes, strided PSUM scales, and configurable SM headroom.
  • Expose upstream shared-expert inference and FP8 expert weights while retaining the fork's existing Python positional arguments and native buffer-slicer tuple formats.
  • Adapt backward output storage and grouped-GEMM K tiles to upstream's interfaces; keep ordinary inference free of host-synchronized config collectives for CUDA graph capture.
  • Retain upstream inference tests in test_mega_moe.py and move the fork's training suite to test_mega_moe_training.py. Add CPU API regressions and a reproducible offline CUDA compilation suite.

Validation passed with GCC 13 and the CUDA/PyTorch development container:

  • C++ extension build and Python packaging.
  • Six CPU compatibility/training regression tests.
  • 26 offline SM100 CUDA variants, including shared experts, FP8/FP8, training saves, 32/240-row tiles, backward, fused weight gradients, and strided PSUM packing.
  • Python syntax and shell syntax checks; both the previous fork tip and upstream target are ancestors of the merge commit.

Draft because this host has no working NVIDIA driver: GPU numerical correctness, distributed synchronization, and performance tests have not run. Shared experts combined with training saves or nondefault route/combine modes are explicitly rejected. Existing raw buffers must be reallocated for the new layout and 1920-token alignment; explicit ring capacities must satisfy the new scheduler's estimate. See docs/fork-upstream-sync.md for compatibility details and validation commands.

Please use a merge commit when landing this sync to retain upstream ancestry.


Note

High Risk
Large merge replacing the JIT/runtime layer and extending MoE, attention, and GEMM APIs on security- and performance-critical GPU paths; callers must re-init JIT, refresh symmetric buffers, and validate env/cache behavior.

Overview
This sync lands upstream DeepGEMM through PR deepseek-ai#432: DeepJIT replaces the in-tree JIT stack and {fmt} (new third-party/deep_jit submodule, local compiler.hpp / cache.hpp removed). init now only takes the library root; DG_JIT_* env vars delegate to DJ_JIT_*, and device/heuristics go through jit / runtime instead of device_runtime.

Attention / indexer: MQA paths are refactored (dtype-driven FP8/FP4, optional schedule_meta, logits cleaning fused in-kernel). New sparse and paged sparse MQA APIs plus metadata helpers and CUDA-graph-friendly workspaces.

GEMM / einsum: Optional alpha on FP8/FP4 and BF16 GEMMs; K-grouped FP4 contiguous GEMM; cuBLASLt NVFP4 GEMM and batched syrk/symm; FP8 BMM/einsum can emit FP8 + UE8M0 SFs. Deterministic mode toggles cuBLASLt vs custom kernels; use_deterministic_algorithms is exposed.

Mega stack: Mega Gate (fused BF16 gate + top-k), Mega mHC (shifted/normal mHC + RMSNorm + optional FP8 outputs), and Mega MoE v3 buffers with shared experts, FP8×FP8 routed weights (mma_type), scheduler-based ring sizing, and get_block_m_for_mega_moe. Paged MQA allows block_kv 32/64/128.

Docs/requirements tighten to CUDA 12.9+, PyTorch 2.3+, C++20 <format>, and libdw-dev.

Reviewed by Cursor Bugbot for commit 38eb89f. Bugbot is set up for automated code reviews on this repo. Configure here.

dfyz and others added 4 commits July 6, 2026 10:30
* Public release 26/09

* Update News
Integrate upstream 66081d4 while preserving the fork training, GeGLU, strided PSUM, and SM-headroom features. Port custom launchers to DeepJIT and adapt backward storage and scheduling to the upstream interfaces.

Validation: extension build, six CPU API regressions, and 26 offline SM100 CUDA variants. GPU numerical and distributed execution remain unverified on this host.
@murphymatt
murphymatt marked this pull request as ready for review September 10, 2026 09:04

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Risk: high. Left a non-blocking comment and did not approve: this upstream DeepJIT/runtime merge changes public GEMM, attention, and MoE APIs with a large blast radius, so it is above the low-risk auto-approval threshold. Human review is needed; no reviewers were assigned.

Open in Web View Automation 

Sent by Cursor Approval Agent: Pull Request Approver

@murphymatt
murphymatt merged commit 8d785f7 into main Sep 12, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants